
IV&V Program 


Acknowledgements 


• Kristin Wortman for inviting the NASA IV&V Program to 
participate in this workshop 

• The NASA IV&V Program (especially Marcus Fisher, Jeff 
Northey and Wes Deadrick) for their contributions to this 
presentation 


September 24, 2014 


2 


NASA IV&V Program 

IV&V Program 

• Mission: To provide our customers assurance that their safety 
and mission-critical software will operate reliably and safely 

• Assurance is focused on: 

- Confidence that the software will do what it is supposed to do 

- Confidence that the software will not do what it is not supposed to do 
(ensure fault avoidance) 

- Confidence that the software will appropriately act/react to/under adverse 
conditions (ensure fault tolerance) 

• Technical Issue Memorandums (TIMs) are provided to the 
developer when evidence suggests that any of the above 
assurance statements cannot be made 

• Risks are proposed to the developer for adoption when evidence 
suggests the development process puts software quality (incl. 
reliability) at risk 

• Reliability is increased when TIMs are resolved or risks are 
mitigated, (assist in fault removal and fault prevention) 
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What is IV&V? 



IV&V Program 



IV&V, as a part of software assurance, plays a role in the overall 
NASA software risk mitigation strategy applied throughout the life 
cycle, to improve the safety and quality of software. 


. ■ 

Software Assurance “umbrella”, described in NASA’s Software 
Assurance Standard (NASA-STD-8739.8) 
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Introduction to IV&V 



IV&V Program 


• Software Verification and Validation (V&V) is a systems 
engineering discipline. 

- V&V is more than testing, just like development is more than coding! 

• The purpose is to help the development organization build quality 
(e.g. reliability) into the software during the software life cycle. 

• Some objectives of performing V&V: 

- Facilitate early detection and correction of software errors, 

- Enhance management insight into process and product risk, 

- Support the software life cycle processes to ensure compliance with 
program performance, schedule, and budget requirements. 

• As part of Software Assurance at NASA, and utilizing IEEE 
standards, IV&V is differentiated from V&V because it is 
managerially, technically, and financially separated from developers 
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IV&V Program 


Introduction to IV&V (cont) 


• IV&V processes determine if development artifacts of a given 
activity conform to the requirements of that activity, and if the 
software artifacts satisfy the intended use and user needs. 

• The validation process provides empirical evidence that 
engineering products: 

- Satisfies system requirements allocated to software 

- Solves the right problem 

- Satisfies the intended use and user needs in expected operational 
environment 

• The verification process provides empirical evidence that 
engineering products: 

- Conform to requirements (e.g., for correctness, completeness, 
consistency, accuracy) during all life cycle phases (e.g., requirements, 
design, code, test), 

- Satisfy standards and best practices, 

- Establish a basis for assessing the completion of each life cycle phase 
and for initiating other life cycle phases. 
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Introduction to IV&V (cont) 

IV&V Program 

• IV&V processes include assessments, analyses, evaluations, 
reviews, inspections, and testing of software artifacts during the 
entire development lifecycle that create evidence 

• Evidence is used to formulate recommendations that improve the quality 
(e.g. reliability) of the system software 

• Evidence is used to make conclusions about the quality (e.g. reliability) of 
the system software 

• Evidence is used to gain insight into the technical progress 

• Evidence is used to judge how thorough you’ve critiqued the system 

• How much evidence -> it is a trade-off between criticality of the system 
being acquired/deployed 

• Life-sustaining subsystems would warrant an evidence package that clearly 
& objectively shows the software will operate safely (or clearly shows that it 
won’t) 

• Data management subsystems may warrant less of an evidence package 

• The amount of evidence needed determines the rigor of the analysis 

• Analytical Rigor is the type and amount of IV&V methods to use for analysis 



September 24, 2014 


Generic Look at IV&V 


IV&V Program 
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Determining the IV&V Assurance Strategy 



IV&V Program 


• The IV&V Program assesses the system to determine 

• which capabilities of the system warrant IV&V analysis 

• the role of software in those capabilities 

• which software elements of the system warrant IV&V analysis 

• The process is called “Portfolio Based Risk Assessment” (PBRA) 

• Results in scores for impact (a measure of the effect of a problem) 
and likelihood (the potential for the existence of errors) for each 
system capability and software element 

• Enables informed decisions to be made regarding: 

• What parts of the system should IV&V work on 

• What analytical rigor should IV&V apply (e.g. dynamic analysis should 
be conducted to thoroughly test the implementation of the protocol 
used for communications) 
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Determining the IV&V Assurance Strategy (cont) 


IV&V Program 
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Formal Analysis 

SMEs conduct formal 
or informal inspections & 
evidence is recorded simply 
as issues 
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Subject Matter Expertise 


• IV&V Processes are applied by individuals with subject 
matter expertise in 

- The analysis method 

- The application of the software under analysis 

- The technologies and methods use to develop the software under 
analysis 

- The types systems that the software under analysis will be integrated 
with 

• IV&V Program Leverages over 20 years of experience 
providing IV&V services to the NASA 
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Current SMSS IV&V Projects 



IV&V Program 



•GOES-R -ICESat-2 "InSight 



•ISS *JWST ‘JPSS -MMS ’MPCV -OSIRIS-REx 



•SPP "SLS 
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IV&V Program 


Past SMSS IV&V Projects 

(not exhaustive) 
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IV&V Program 


Summary 


• IV&V helps build reliability into SMSSs by 

- Increasing the likelihood of discovering and removing critical defects 
throughout the development lifecycle 

- Focusing analyses on ensuring correct and complete fault avoidance, 
and fault tolerance 

- Applying best practices in its assessments, analyses, evaluations, 
reviews, inspections, and testing of software artifacts 
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